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DECODING APPARATUS AND METHOD, AND STORAGE 
MEDIUM STORING DECODING PROCESSING PROGRAM OF THE SAME 



BACKGROUND OF THE INVENTION 
5 Field of the Invention 

The present invention relates to a data decoding 
apparatus and method,, and a storage medium and, more 
particularly, to decoding processing of decoding a 
bitstream such as coded and multiplexed image and audio 

10 signals in a digital TV reception apparatus or a 
digital storage medium reproduction apparatus. 
Related Background Art 

Fig. 1 shows a conventional decoding apparatus for 
decoding a coded bitstream. Fig. 1 shows the 

15 arrangement of a digital broadcasting receiver that has 
an accounting function. Referring to Fig. 1, the 
apparatus comprises a bitstream input terminal 101 for 
receiving a coded bitstream, descrambling circuit 102, 
and separation circuit 103 for separating multiplexed 

20 data. 

The apparatus also has an audio signal decoding 
circuit 104 for decoding a separated audio signal, a 
video signal decoding circuit 105 for decoding a 
separated video signal, a CPU 106, an IC card 107, and 
25 a remote controller 108. The operation of the 

conventional digital broadcasting receiver will be 
described below with reference to Fig. 1. 
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A bitstream received by a tuner (not shown) in 
Fig. 1 is input from the input terminal 101 to the 
apparatus. As the coding scheme, so-called MPEG 
(Moving Picture Experts Group) 2 is used. 
5 In this input stream, normally, a coded audio 

signal and video signal are multiplexed and scrambled 
to permit only a specific user to view the contents. 

A method of confirming viewing permission will be 
described later. If viewing is permitted, the stream 
10 input to the descrambling circuit 102 is descrambled by 
a descrambling key given by the CPU 106 and input to 
the separation circuit 103. 

The separation circuit 103 separates the stream 
into an audio signal stream and a video signal stream, 
15 which are supplied to the decoding circuits 104 and 

105, respectively, decoded, and displayed on a monitor 
(not shown). Control lines from the CPU 106 are 
connected to the audio signal decoding circuit 104 and 
video signal decoding circuit 105 to control 
20 stereo/multi -lingual data or display information on the 
screen. 

A method of confirming viewing permission 
(authentication method) will be described. 

A subscribed program ID and descrambling key are 
25 stored in the IC card 107 in advance. To watch the 
program, the CPU 106 accesses the IC card 107 and 
confirms viewing permission on the basis of whether the 
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program ID stored in the IC card and the program of the 
input stream coincide with each other. In this case, 
programs that are not subscribed in advance cannot be 
watched . 

5 Conventionally, however, in any schemes, viewing 

is limited in units of channels or in units of 
programs. In addition, only ON/OFF-control can be 
performed. For this reason, it is difficult to provide 
higher-quality video data in accordance with 

10 subscription or provide an interactive program. 

In recent years, large-capacity media such as DVD 
are becoming popular, and not only fixed control for 
reproduction/non-reproduction but also a variety of 
flexible use forms that take advantage of the large 

15 capacity are required. However, such a service is hard 
to realize. 

SUMMARY OF THE INVENTION 

The present invention has been made in 

20 consideration of the above situation, and has as its 

object to provide a decoding apparatus and method and a 
storage medium storing a decoding processing program, 
which permit to set a multi-level reproduction 
processing of an object when bitstream which is coded 

25 and multiplexed in units of a plurality of objects is 
decoded . 



In order to achieve the- above object, according to 
an aspect of the present invention, there is provided a 
decoding apparatus/method in which a bitstream obtained 
by coding a plurality of object data in units of 
5 objects and multiplexing the coded data is input, coded 
data of each object is separated from the bitstream, a 
predetermined object is selected from the plurality of 
objects contained in the bitstream, the coded data of 
the selected object is decoded and object data is 

10 output, and the decoded object data is synthesized. 

According to another aspect of the present 
invention, there is provided a computer-readable 
storage medium which stores a decoding program 
comprising input processing of inputting a bitstream 

15 obtained by coding a plurality of object data in units 
of objects and multiplexing the coded data, separation 
processing of separating coded data of each object from 
the bitstream, selection processing of selecting a 
predetermined object from the plurality of objects 

20 contained in the bitstream, decoding processing of 
decoding the coded data of the selected object and 
outputting object data, and synthesis processing of 
synthesizing the decoded object data. 

Other objects, features and advantages of the 

25 invention become apparent from the following detailed 
description taken in conjunction with the accompanying 
drawings . 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram showing the arrangement 
of a conventional decoding apparatus; 

Fig. 2 is a block diagram showing the arrangement 
5 of a decoding apparatus according to an embodiment; 

Figs. 3A, 3B, 3C and 3D are views for explaining 
the operation of a priority decoder; and 

Fig. 4 is a block diagram showing details of a 
synthesization circuit 210 shown in Fig. 1. 

10 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

In recent years, MPEG4 has received a great deal 
of attention as a new coding scheme. As some 
characteristic features of MPEG4, it has scalability in 

15 space and time, and additionally, it divides video data 
into a plurality of objects (e.g., background and 
persons), and codes and multiplexes respective data, 
unlike the conventional MPEG2. 

As some other characteristic features, coding and 

20 multiplex corresponding to CG or animation are 

possible, and the error resilience is high. This 
embodiment places an emphasis on the processing of a 
plurality of objects. In this embodiment, an MPEG4 
bitstream is decoded. 

25 Fig. 2 is a block diagram showing the arrangement 

of a decoding apparatus according to this embodiment. 
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Referring to Fig. 2, the decoding apparatus has an 
antenna 201 for receiving a broadcasting signal, and a 
tuner 202 for demodulating the reception signal 
received by the antenna 201 to obtain a coded 
5 bitstream. 

The apparatus also has a descrambling circuit 203, 
a separation circuit 204 for separating multiplexed 
data, and an audio signal decoding circuit 205 for 
decoding a separated audio signal. 

10 Decoding circuits 206 to 208 decode separated VOP 

(Video Object Plane) signals, respectively. A scene 
description decoding circuit 209 decodes a scene 
description. A synthesization circuit 210 synthesizes 
the audio signal and VOP signals. 

15 The decoding apparatus also has switches 211 to 

213, a priority decoder 214, a communication circuit 
215 fox connecting the apparatus to a communication 
line such as a telephone line or a line of the Internet 
or cable TV, a CPU 216, an IC card 217, a remote 

20 controller 218, and a display apparatus 219 for audio 
and image output. 

The processing operation of the decoding apparatus 
of this embodiment having the above-described 
arrangement will be described below. 

25 An MPEG4 bitstream obtained through the antenna 

201 and tuner 202 or an MPEG4 bitstream obtained 
through the communication circuit 215 is selectively 
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input to the descr ambling circuit 203. If viewing is 
permitted, a descrambling key is supplied from the CPU 
216, and the descrambling circuit 203 descrambles the 
bit stream. 

5 If viewing is not permitted, video data for 

notifying the user that viewing is not permitted is 
generated by the CPU 216 and displayed on the display 
apparatus 219 through the synthesization circuit 210. 

A method of authenticating viewing permission will 
10 be described below. 

First, the user subscribes to a management center 
(a company that supplies bitstreams or a management 
company entrusted by the company) through the 
communication circuit 215 in advance, downloads a 
15 program ID and descrambling key from the management 
center, and stores them in the IC card 217. 

To manage and protect intellectual properties such 
as a copyright contained in the MPEG4 bitstream, the 
bitstream contains IPMP (Intellectual Property and 
20 Management Protection) data. The IPMP data contains 

the same program ID as that stored in the IC card 217. 

The CPU 216 extracts the IPMP data from the 
bitstream input to the descrambling circuit 203 and 
detects the program ID. 
25 Next, the CPU 216 reads out the program ID stored 

in the IC card 217 and authenticates whether the 
readout program ID coincides with the program ID 
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detected from the IPMP data. If the two program IDs 
coincide with each other, it is determined that viewing 
is permitted- In this case, the program cannot be 
watched unless the user subscribes in advance. 
5 In addition to the above method, a method that 

charges for viewing can also be used to obtain viewing 
permission. 

In this case, a descrambling key used for 
descrambling and communication information (e.g., a 

10 URL) for contact with the management center are 

contained in the IPMP data. The viewer can descramble 
the data using the descrambling key contained in the 
IPMP data at any time. When the descrambling key is 
used, personal information and viewing record stored in 

15 the IC card 217 are transmitted to the management 

center on the basis of the communication information, 
and charges incurred are billed later. That is, a 
pay-per-view scheme is employed. 

In this embodiment, a viewing age limiting 

20 function is prepared. Unless a password is input 

through the remote controller 218, a specific program 
(e.g., an adult program ) cannot be watched 
(descrambled) even if charges have been payed. 

Processing after viewing is permitted and the data 

25 is descrambled by the above-described method will be 
described next. 
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The descr ambled stream is separated by the 
separation circuit 204 into streams such as audio, 
VOP0, VOP1, VOP2, and scene description streams. VOP 
is the basic unit of a video image in MPEG4 and 
5 actually represents the instantaneous value (image) of 
a video ob j ect . 

MPEG4 has a number of layers including an upper 
layer called a VOL (Video Object Layer) in which, e.g., 
a VOP is expanded in space or time to increase 
10 resolution, and a VO (Video Object) that is a 

collection of VOLs. In this embodiment, a basic VOP 
will be exemplified for the descriptive convenience. 
The number of VOPs is limited to three in this example. 
The streams separated by the separation circuit 
15 204 are decoded by the audio signal decoding circuit 

205, VOP decoding circuits 206, 207, and 208, and scene 
description decoding circuit 209, synthesized by the 
synthesization circuit 210, and output to the display 
apparatus 219. 

20 The scene description is a description using a 

language, which positions VOPs divided into objects in 
space and synchronizes them in time to obtain one video 
image (audio data is also regarded as one object and 
synchronized in time). 

25 The reason why not only spatial positioning but 

also temporal synchronization is necessary is that each 
object is influenced by time shift due to multiplex or 
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variations due -fco decoding delay. Actually, buffers 
must be prepared on the input and outputs sides of each 
decoding circuit in Fig. 2, although they are omitted 
in Fig. 2 for the illustrative convenience. 

Referring back to Fig. 2, the switches 211, 212, 
and 213 determine whether VOP decoding outputs are to 
be output to the synthesization circuit 210. When 
these switches are turned off, an output different from 
the original MPEG4 video output is obtained. 

More specifically, when a certain switch is turned 
off, an object corresponding to the switch is not 
displayed on the screen. The switches 211, 212, and 
213 are ON/OFF-controlled by the priority decoder 214. 

The priority decoder 214 functions as shown in 
Figs. 3A to 3D. That is, the priority decoder 214 
ON/OFF-controls the switches 211 to 213 in accordance 
with a value output from the CPU 216. When priority 

decoder VOP0, 1, 2, are assigned sequentially from 

an important video object in advance before coding 
video data, an integer value 0, 1, 2, or 3 is supplied 
in accordance with the subscription of the user, 
thereby providing a service by preparing a multiple of 
resolutions to one program. 

As the subscription procedures, only one extra 
integer value for object selection is stored in the IC 
card in addition to the program ID and descrambling 
key. In pay-per-view as well, the numerical value need 
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only be added to viewing information to be transmitted 
to the management center . 

An explicit example of discrimination of the 
service will be described with reference to the block 
diagram of Fig. 4. Fig. 4 shows a specific example of 
the synthesization circuit 210 shown in Fig. 2. A VOP 
in Fig. 4 corresponds to a VOL in which the VOP is 
expanded to increase resolution. VOP0 is an image 
having the lowest resolution. If VOP1 and VOP2 are not 
given, an image obtained by simply expanding a 
subwindow in the vertical and horizontal directions is 
obtained as a video output. 

V0P1 or V0P2 is high-resolution image information. 
When V0P1 or VOP2 is supplied to the synthesization 
circuit 210, an image with higher resolution is 
obtained as a video output. Hence, images of a 
plurality of image quality levels can be provided in 
units of subscription contents of users in 
correspondence with a common bitstream in accordance 
with integer values according to the subscriptions. 

The numerical values shown in Figs. 3A to 3D will 
be exemplified. A user who has subscribed to "3" in 
Fig. 3D receives a service with the highest image 
quality. Referring to only Figs. 3A to 3D, it appears 
this system can be implemented using MPEG2 hierarchical 
coding. However, herein the basic concept is "video 
object", and the resolution of only an object (part) on 
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the window can be changed. MPEG2 permits only 
hierarchical coding targeting the entire picture. 

A supplementary explanation will be done below in 
association with this embodiment. In this embodiment, 
the priority of VOPs is determined using integer 
values. Instead, command mnemonic or the like may be 
used. Alternatively, the current level may be stored, 
and the level may be changed by an UP /DOWN command. 

In this embodiment, the integer values coincide 
with the number of VOPs to be decoded. Since the 
number of VOPs always varies, a ratio to the maximum 
allowable number of VOPs (e.g., 50% decoding or 20% 
decoding) may be used as an index. 

Referring to Fig. 2, the switches 211, 212, and 
213 are controlled to determine the priority of the 
VOPs. However, the VOP decoding circuits 206, 207, and 
208 may be controlled to inhibit that decoding is 
performed selectively. 

Instead of using the priority decoder 214, direct 
control may be done by the CPU 216 (decoding by 
software). In this embodiment, audio objects have not 
been mentioned. For example, for an application in 
which a plurality of audio data are multiplexed in a 
one-to-one correspondence with VOPs, processing for 
VOPs in this embodiment can be applied. 

When this embodiment is applied while multiplexing 
a plurality of scene descriptions, a service with a 
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more flexible hierarchical structure can be provided as 
compared to this embodiment in which priority is given 
to VOPs. In this embodiment, the description has been 
made assuming broadcasting- However, this embodiment 
can also be applied to package media such as a DVD. 

This embodiment can also be applied to an 
interactive (bidirectional) application such as a quiz 
program because ON/OFF control and other control can be 
performed in units of objects. 

As described above, according to this embodiment, 
the precision when a bitstream coded and multiplexed in 
units of a plurality of objects is separated into 
object streams and decoded can be controlled to a 
plurality of levels. For this reason, a variety of 
flexible services can be realized for an input 
bitstream. For example, in a broadcasting medium, 
viewing can be conventionally limited in units of 
channels or in units of programs. In addition, only 
ON/OFF control can be performed. However, a 
high-quality image can be provided in accordance with 
the subscription even for one program. 

In addition, additional circuits or additional 
software for priority assignment can be minimized. 
When priority is given to VOPs on the coding side, any 
large failure can be avoided even in an apparatus 
having a low ability and capable of decoding only two 
VOPs on the receiving side. 
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The present invention may be applied to a system 
constituted by a plurality of devices (e.g., a host 
computer or an interface device) or an apparatus 
comprising a single device. 

The present invention also incorporates an 
arrangement in which various devices are operated such 
that the function of the embodiment is realized by 
supplying software program codes for realizing the 
function of the above embodiment to the computer in the 
apparatus or system connected to the various devices, 
and causing the various devices to operate in 
accordance with the program stored in the computer ( CPU 
or MPU) of the system or apparatus. 

In this case, the software program codes realize 
the function of the above-described embodiment by 
themselves, and the program codes or means for 
supplying the program codes to the computer, e.g., a 
storage medium storing the program codes constitute the 
present invention. As a storage medium storing the 
program codes, a floppy disk, a hard disk, an optical 
disk, a magnetooptical disk, a CD-ROM, a magnetic tape, 
a nonvolatile memory card, a ROM, or the like can be 
used. 

The function of the above-described embodiment is 
realized not only when the supplied program codes are 
executed by the computer but also when the OS 
(Operating System) running on the computer realizes the 
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function of the above embodiment in collaboration with 
those program codes. 

The present invention also incorporates an 
arrangement in which the function of the 
above-described embodiments is realized when the 
supplied program codes are written in the memory of a 
function expansion board inserted into the computer or 
a function expansion unit connected to the computer, 
and the CPU of the function expansion board or function 
expansion unit performs part or all of actual 
processing on the basis of the instructions of the 
program codes. 

In other words, the foregoing description of 
embodiments has been given for illustrative purposes 
only and not to be constructed as imposing any 
limitation in every respect . 

The scope of the invention is, therefore, to be 
determined solely by the following claims and not 
limited by the text of the specifications and 
alternations made within a scope equivalent to the 
scope of the claims fall within the true spirit and 
scope of the invention. 
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WHAT IS CLAIMED IS: 

1. A decoding apparatus comprising: 

a) input means for inputting a bitstream obtained 
by coding a plurality of object data in units of 
objects and multiplexing the coded data; 

b) separation means for separating coded data of 
each object from the bitstream; 

c) selection means for selecting a predetermined 
object from the plurality of objects contained in the 
bitstream; 

d) decoding means for decoding the coded data of 
the object selected by said selection means and 
outputting object data; and 

e) synthesis means for synthesizing the object 
data decoded by said decoding means. 

2. An apparatus according to claim 1, wherein 
the bitstream is a bitstream complying with MPEG4. 

3. An apparatus according to claim 1, wherein 
the bitstream input to said input means is 

scrambled, and said input means comprises descrambling 
means for descrambling the scrambled bitstream. 



4. An apparatus according to claim 3, wherein 
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the bit stream contains IPMP data that is not 
scrambled, and said descrambling means descrambles the 
scrambled bitstream in accordance with the IPMP data. 

5. An apparatus according to claim 3, wherein 
said apparatus further comprises read means for 

reading descrambling data for descrambling the 
scrambled data, the descrambling data being stored in 
an IC card, and said descrambling means descrambles the 
scrambled bitstream in accordance with the descrambling 
data read by said read means. 

6. An apparatus according to claim 1, wherein 
said apparatus further comprises read means for 

reading selection data for selecting the object, the 
selection data being stored in an IC card, and said 
selection means selects the predetermined object from 
the plurality of objects in accordance with the 
selection data read by said read means. 

7. An apparatus according to claim 1, wherein 
the plurality of objects include at least a video 

object . 

8. An apparatus according to claim 7, wherein 
the plurality of objects include at least an audio 

ob j ect . 
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9. An apparatus according to claim 8, wherein 
the plurality of objects include at least a scene 
description object. 

10. An apparatus according to claim 1, further 
comprising 

monitor means for monitoring the object data 
synthesized by said synthesis means. 

11. An apparatus according to claim 1, further 
comprising 

communication means for performing data 
communication with an external device, said 
communication device transmitting, to said external 
device, information representing that the bitstream is 
decoded . 

12. An apparatus according to claim 11, wherein 
said communication means performs data 

communication through the Internet. 

13. A decoding method comprising the steps of: 
inputting a bitstream obtained by coding a 

plurality of object data in units of objects and 
multiplexing the coded data; 

separating coded data of each object from the 
bitstream; 
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selecting a predetermined object from the 
plurality of objects contained in the bitstream; 

decoding the coded data of the selected object and 
outputting object data; and 
5 synthesizing the decoded object data. 



14. A computer-readable storage medium which 
stores a program, said program comprising steps of: 

a ) input processing of inputting a bitstream 

10 obtained by coding a plurality of object data in units 
of objects and multiplexing the coded data; 

b) separation processing of separating coded data 
of each object from the bitstream; 

c) selection processing of selecting a 

15 predetermined object from the plurality of objects 
contained in the bitstream; 

d) decoding processing of decoding the coded data 
of the selected object and outputting object data; and 

e) synthesis processing of synthesizing the 
20 decoded object data. 
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ABSTRACT OF THE DISCLOSURE 

This invention is to provide a decoding 
apparatus/method in which a bitstream obtained by- 
coding a plurality of object data in units of objects 
5 and multiplexing the coded data is input, coded data of 
each object is separated from the bitstream, a 
predetermined object is selected from the plurality of 
objects contained in the bitstream, the coded data of 
the object selected by selection means is decoded, 
10 object data is output, and the decoded object data is 
synthesized . 
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I hereby appoint the practitioners associated with the firm and Customer Number provided below to prosecute this application and 
to transact all business in the Patent and Trademark Office connected therewith, and direct that all correspondence be addressed to the 
address associated with that Customer Number: 

FITZPATRICK, CELLA, HARPER & SCINTO 
Customer Number: 05514 



I hereby declare that all statements made herein of my own knowledge are true and that all statements made on information and 
belief are believed to be true; and further that these statements were made with the knowledge that willful false statements and the like so 
made are punishable by fine or imprisonment, or both, under Section 1001 of Title 1 8 of the United States Code and that such willful false 
statements may jeopardize the validity of the application or any patent issued thereon. 

Full Name of Sole or First Inventor YUKINORI YAMAMOTO 



Inventor's signature 

Date Citizen/Subject of JAPAN 

Residence 691-1-101, Shinmaruko - machi , Nakahara-ku 

Kawasaki - shi , Kanaqawa-ken, Japan 

Post office Address c/o Canon Kabushiki Kaisha, 



3 0-2, Shimomaruko 3-chome, Ohta-ku, Tokyo, Japan 



NY MAIN 60385vl 



